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ABSTRACT 

The conceptual framework for this research describes the processes faculty use to create an online 
course that meets learning outcomes while creating a positive learning experience for the online 
student. This involves acceptance of technology to create a course structured for inductive 
learning in addition to traditional deductive learning. The presence of faculty innovation is 
demonstrated by the design of inductive learning exercises. Innovative professors must continue 
to enhance the quality of their online delivery in an effort to achieve the same educational 
outcomes acquired in a traditional classroom. However, many professors are reluctant to 
embrace online delivery software and technology designed to enhance the classroom. This 
research identifies innovation, technology acceptance, inductive learning, and quality 
enhancement as measures online quality in an academic environment. The assessment instrument 
for this research is the Rubric for Online Competencies and Standards (ROCS) developed by 
Hodges University. The ROCS consist of twenty-eight quality indicators in six categories 
extracted from a combination of regional standards from accreditation bodies including, but not 
limited to: SACS, Sloan-C, and EDUCAUSE. 

Keywords: Faculty Innovativeness; Technology; Inductive Learning; Distance Education; Online Course; 

Pedagogy; Learning Community 


INTRODUCTION 



his research examines innovativeness of faculty in regards to acceptance of technology to provide 
inductive learning opportunities when teaching online. Today’s online delivery software and 
supporting technology provides faculty numerous tools necessary to deliver an online course that 
meets the rigorous academic standards found in a traditional classroom offering. However, there is significant 
research indicating that faculty remains reluctant to use the necessary technology to create learning opportunities 
that will enhance his/her course. 


It is hard to imagine any university Dean allowing a faculty member to remain on staff if they delivered a 
traditional classroom course offering virtually no collaboration, inductive learning, faculty lectures, interactive 
activities, or communication between the student and professor. Unfortunately, while the majority of online 
offerings maintain very high standards, research indicates that too many courses are below average. Academic 
institutions must assess online courses and provide professional development to eliminate substandard practices. 
This paper outlines an assessment process utilized by Hodges University of Naples Florida. 

This research examines the literature on innovation, inductive learning, and acceptance of technology in 
online instruction. The analysis section of this research outlines a peer-to-peer review process conducted by Hodges 
University to assess how professors utilize technology to create inductive learning methods i.e., completing projects. 
This research is in the early stages and this paper will serve as the starting point for a detailed study on faculty 
innovation in regards to technology acceptance and inductive learning techniques. 
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Courses evaluated in this research use the Blackboard online delivery software. Research indicates that 
Blackboard is one of the leading web based delivery software systems utilized by colleges and universities. Studies 
indicate that Blackboard significantly improves the delivery of online courses (Roberts-DeGennaro et al., 2005; 
Carnevale, 2006; Larkin & Benson, 2005; Jefferies et al., 2003). Although, Blackboard provides the basic course 
enhancement tools, it is up to the university administration and faculty to utilize its capabilities and determine which 
enhancements are necessary to deliver the required learning outcomes. While Blackboard provides modules to 
guide students through the course, anarchy could prevail in the course if guidelines and structure are not provided by 
competent faculty. 

Conceptual Framework 

The conceptual framework for this research describes the processes faculty use to create an online course 
that meets program outcomes while delivering a positive learning experience for the online student. This involves 
acceptance of technology to create a course structured for inductive learning in addition to traditional deductive 
learning i.e., applying concepts by examination. The presence of faculty innovation is demonstrated by the design 
and execution of inductive learning exercises. The variables listed in the conceptual framework are reviewed in the 
literature review and analyzed through the peer-to-peer review process discussed in the analysis section of this 
paper. 


Conceptual Framework 



Technology Acceptance 

It is difficult to implement online learning without the acceptance of new technology by institution faculty. 
Masrom (2007) defined technology acceptance as “an individual’s psychological state with regard to his or her 
voluntary or intended use of a particular technology.” The Technology Acceptance Model (TAM) is a tool utilized 
by Masrom to verify end-users acceptance of technology. According to Maslin’s research there is a significant 
reluctance on the part of faculty to accept technology even though online instruction is a major component in most 
universities. 

Waheed and Farooq (2010) proposed an extended TAM model to address the willingness of faculty to 
accept online technology. Their study determined that four factors influenced professors to accept technology 
designed for online learning. Their study included faculty at an online university and concluded that perceived ease 
of use, perceived usefulness, facilitating conditions, and computer efficacy were determining factors in faculty 
willingness to implement new e-learning systems. They discovered that tenure played a significant role in 
acceptance with faculty teaching twenty years or more displaying the most reluctance to adopt technology. Their 
research also revealed that professors were concerned that students would know more about the technology than 
they do, thus causing discomfort for the faculty member. Research by Liaw et al., (2007) indicated that training 
must be provided to ensure that faculty possesses the skills necessary to respond to student questions. Other 
research concluded that workload and time played a significant role in faculty acceptance of online technology 
(Tabata & Johnsrud, 2008). 
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A study by Lin et al. (2010) examined use and resistance of technology in universities. Their study 
included 1,022 faculty and administration from various universities. This research utilized the conceptual 
framework of technology and structural enactment developed by Orlikowski (2000). This framework concluded that 
structure was not a physical entity, but institutional knowledge activated by action on the part of staff and faculty. 
Wenger’s community of practice is another avenue of scholarly support for this concept (Wenger et al., 2002). Once 
it is established that structure is not materialistic, planners can create a strategy to enhance the online classroom by 
implementing technological changes that enhance instruction. Lin’s study determined that structure enables faculty 
to enhance quality in the online classroom, but can also constrain instructional efforts if the premise of structure 
being an institutionalized culture is not accepted. 

Faculty interaction with Blackboard and any supporting technology is determined by the course strategy 
being augmented by human action. Faculty members must be intricately involved in every aspect in development of 
online courses. Acceptance of technology is critical in an academic setting as technological capabilities define the 
institutions ability to sustain a competitive advantage. Since most educational institutions share the same 
technology assets, it is imperative that a competitive advantage is gained through the method in which the 
technology is used. 

Delivering a quality course utilizing inductive learning techniques created by innovative faculty with an 
emphasis on technology is a key to success. At Hodges University, emphasis is placed on the availability of assets 
while still maintaining academic freedom. Each course is unique and treated as such with faculty members 
maintaining the freedom to create a course that meets the learning outcomes required. The distance education staff 
at Hodges University provides a Blackboard template for continuity, training, and support assets, but otherwise 
allows faculty to be innovative. The peer-to-peer assessment instrument addressed in this research analyzes success 
and identifies professional development opportunities based on results. 

Faculty Innovation 

Innovation on the part of online faculty is based on the open innovation concept identified in a book by 
Chesbrough (2006). Open innovation is defined as “...the use of purposive inflows and outflows of knowledge to 
accelerate internal innovation, and expand the markets for external use of innovation, respectively” (Chesbrough et 
al., 2006, p. 32). This is the type of innovation required of faculty as new technology is combined with inductive 
learning techniques to create quality in online learning. In an academic setting, open innovation provides 
discoveries, findings, inventions, and knowledge. Open innovation provides the link between strategies, technology 
and inductive exercises that produce competencies demanded by current students. To compete in today’s 
educational marketplace, universities cannot rely on traditional methods. Universities must engage in new idea 
generation by beginning in the classroom with meaningful inductive learning exercises that engage students and 
produce a viable product at the end of the course. This concept is also supported by educational research on 
“constructivism.” Social Cognitive theorists such as Bandura also support the social construction of knowledge 
through group work and instructor interaction (Morrison et al., 2007, p. 158-159). 

Eynon (2008) conducted a study of forty-one staff members at two British universities to determine the 
motivation issues guiding institutions utilizing information technology in the classroom. Results indicated that 
positive motivators included supporting existing resources, encouraging independent learning, providing additional 
resources to professors, and providing an experience for students. Additionally, motivational factors included in the 
study for not utilizing technology in the classroom included time issues, technical support, constraints with the 
course, roles, and structural issues. Overall, university staff encouraged the use of technology because of profits and 
flexibility, while faculty resisted due to various changes necessary to adapt the course environment to include 
technology. Faculty embracing technology cited innovation, flexibility, and enhanced teaching capabilities. The 
adoption of technology in the classroom depends on the level of faculty innovation combined with perceived support 
from university administration. 

Inductive Learning 

Another important variable in development of an enhanced online course is utilization of an inductive 
learning process. Traditional courses are more often than not taught deductively by professor’s utilizing lectures. 
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models, theory, and examinations while students apply the material to real-world scenarios if possible. Students are 
not motivated through doing, but rather by exams and possible future uses. Inductive learning techniques offer 
students motivation through the challenge of completing a project with material delivered in the online classroom, 
but requiring them to complete an applied project. Students solve a case, interpret data, or apply knowledge to real- 
world scenarios. 

Prince and Felder (2007) define inductive learning as “Instruction that begins with a challenge for which 
the required knowledge has not been previously provided technically qualifies as inquiry-based learning, and the 
scope of the inquiry may vary from a portion of a single lecture to a major term project. In this sense, all inductive 
methods are variants of inquiry, differing essentially in the nature of the challenge and the type and degree of 
support provided by the instructor” (p. 15). Prince and Felder advocates that inquiry learning techniques or 
inductive learning improves academic achievement and better prepare students for real-world issues. This thought is 
also supported by the new Bloom’s Taxonomy through creation (formerly synthesis) and evaluation (Krathwohl, 
2002). 


According to Prince and Felder (2007), students involved in case-based learning, problem- based learning, 
discovery learning, just-in-time teaching, and project-based learning through questions supported with materials and 
a challenge to solve a real-world problem will develop the skills required to master subject materials. Research on 
inductive learning supports students adopting a deeper approach to learning when this approach is used (Ramsden 
2003; Norman & Schmidt 1992). The peer-to-peer process utilized by Hodges University measures inductive 
learning techniques in several areas and indicates how faculty members apply these techniques through technology. 

Enhanced Online Classroom 

Research indicates that computer technology provides instructional technology that enhances the faculty’s 
ability to deliver a quality course (Eynon, 2008). To this end, it is important that university administration provide 
the infrastructure to support faculty innovation while at the same time measure delivery to ensure faculty 
engagement in the use of technology. Research by Inoue and Buczynski (2011) points out that inductive learning is 
dependent upon instructors challenging students to be active participants in the learning process rather that accepting 
only what the professor provides. Therefore, faculty must be innovative in the use of technology and the process of 
course design to ensure students are actively engaged in learning activities. 

When participating in inductive learning, students are encouraged to design their own solutions to complex 
problems instead of the professor providing detailed step-by-step procedures. The key elements in this practice are 
conjecturing, invention, and problem solving as students develop multi-strategy solutions to workplace problems 
(Van de Walle, 2004). Inductive learning can evolve from guided inquiry with the teacher leading to student 
centered learning or open inquiry with the student accepting responsibility for solution to the problem. The goal is 
student involvement in solving a real-world issue. 

Research indicates that delivering inductive learning in an online environment requires faculty to: 1) begin 
with a problem that is meaningful and applicable to real-world situations, 2) allow students to apply current 
knowledge with new knowledge by asking open-ended questions, 3) provide guidance by placing a priority on 
important elements of the problem, 4) promote self-direction and collaboration by encouraging students to be 
creative, and 5) provide opportunities for students to connect theory to practice (Ormrod, 2003; Woolfolk, 2006). 
Following these steps and providing assessments that measure progress will create an inductive learning 
environment. Successful outcomes are dependent upon faculty innovation and use of available technology. Inoue 
and Buczynski (2011) contends that a stumbling block to success is when students ask an open-ended question and 
the faculty member is not prepared to provide guidance. 

The peer-to-peer review process at Hodges University provides evaluation by fellow faculty members in an 
effort to continuously improve the online program. The analysis section below describes results from the FY2011 
evaluation. 


Copyright by author(s); CC-BY 


10 


The Clute Institute 



Contemporary Issues In Education Research - First Quarter 2014 _ Volume 7, Number 1 

ANALYSIS 

As noted earlier, research by Orlikowski (2000) determined that, when measuring innovation, 
organizational structure was not a physical entity, but institutional knowledge activated by action on the part of staff 
and faculty. At Hodges University, administration officials have worked diligently to ensure courses delivered 
online meet university standards and enhance the institutional knowledge in regards to online learning. Knowledge 
enhancement is a philosophy instituted by action, expectations, and assessment. In addition to the traditional end-of- 
term evaluations and faculty evaluations, Hodges University employs other methods to assess the pedagogy in the 
online program. Methods such as student information sheet, expectations essay, questionnaire, early term evaluation 
and others provides data for administrators to gauge faculty responsiveness. The main assessment instrument for 
measuring technology acceptance, faculty innovation, inductive learning techniques, and delivery of a quality online 
program is the Rubric for Online Competencies and Standards (ROCS). 

It is the annual review of the ROCS evaluations that drives discussions among colleagues on the Distance 
Education Committee toward a discovery and adoption process. Two theories of organizational workings help to 
explain this process. First, the community of practice theory of social learning (Wenger, 1998) explains how 
professional development and collaboration play key roles in organizational learning, a primary goal of assessment. 
In this case, learning within a community brought about mutual understanding. Second, sense making (Weick, 
1995) describes the process through which the results of assessment and self-reflection can be transformed into 
organizational goals. Mutual understanding, structure, and process together form the conditions in which the roles 
of these higher education practitioners intersect to propel distance education toward its future. 

Data Collection 

The ROCS assessment instrument has evolved since 2007 with significant improvement to the process each 
year. The ROCS consist of twenty-eight quality indicators in six categories extracted from a combination of 
regional standards from accreditation bodies including, but not limited to: SACS, Sloan-C, and EDUCAUSE. For 
the purpose of this research, we will concentrate on indicators of technology acceptance, inductive learning, and 
faculty innovation. 

Since the inception of the ROCS assessment instrument, each online faculty member is required to submit a 
self-evaluation annually. Supervisors review this self-assessment and complete an evaluation of one course using 
the same instrument. This becomes part of the annual faculty review. Initially, this process was augmented with a 
university peer-to-peer review where faculty evaluated each other. To improve quality, distance education staff 
enacted the current version of the peer-to-peer review by selecting eighteen courses representing lower division, 
upper division, and graduate level offerings. A team approach was employed where faculty and distance education 
staff teamed together to evaluate the six categories of the ROCS. Each team was assigned one category to 
strengthen reliability. 

Courses were randomly selected from the Fall 2011 term to compare to the same term in previous 
evaluations. All twenty-eight indicators of exemplary practice in an online course were evaluated on a scale of zero 
(Not Evident) to five (Exemplary). The combined statistical analysis coupled with historical data provided the 
university with a training needs assessment. Several training initiatives were implemented or improved including 
instructional design courses. Blackboard training, best practices, user groups, and others. At this printing, the 2012 
review is underway with the hope that significant improvement will be noted. 

Data Analysis/Discussion 

Inter-rater reliability (simple correlation) was calculated for each of the six teams of two, focusing upon a 
related group of ROCS indicators in the 2011 assessment. It was expected that scores for 2011 would be slightly 
lower due to improved evaluation processes. There were some pleasant surprises and some training opportunities 
noted. Score results are shown in Tables l-6a and correlations between team members’ scores are displayed in 
Tables l-6b. 
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Overall, there was a strong correlation between scorers in two teams and a moderate correlation on one 
team, indicating that there was agreement among half of all team members on how to apply the indicators to 
individual online courses. 2011 score results may be an indicator of closer scrutiny from evaluators and should 
improve in 2012. 

Learner Support 

All indicators in this category were lower than in previous reviews. In fact, the average for the category for 
Fall 2011 was fully one point lower (on a scale of 0-5). The lowest indicator was 1 . Links to library and online 
learning resources. This was something of a mystery, as each course contains a menu button for Library that is 
populated each term by library staff. However, with a correlation of .74, this team was mostly in agreement about 
how to apply the indicators to individual courses. 


Table la: Four-Year Trend for Scores in Learner Support and Resources 


Category 

ROCS 

No. 

Indicator 

2007 

Fall 

2008 

Fall 

2010 

Fall 

2011 

Fall 


1 

Links to library and online learning resources 

3.7 

3.3 

3.7 

2.7 

Learner 

2 

Course and program resources and contact information 

3.3 

3.4 

4.3 

2.9 

Support 

3 

Range of resources supporting course content 

3.1 

3.7 

3.9 

3.1 


Category Average 

3.4 

3.5 

3.9 

2.9 


T able lb: Correlations between Scorers in Learner Support and Resources in Fall 201 1 


Learner Support and Resources (Strong Correlation) 

0.74 

1 

Links to library and online learning resources 

0.60 

2 

Course and program resources and contact information 

0.74 

3 

Range of resources supporting course content 

0.88 


Online Organization & Design 

This category shows evidence that support elements for the online program are a training opportunity. 
While each indicator in the category was rated lower than in previous reviews, the indicator that really stood out on 
the chart was <5. ADA Accessibility. The extent of non-compliance on this indicator was a surprise to the reviewers. 
Corrective actions included the adoption of a set of standards for ADA compliance and plans for faculty professional 
development to meet those standards. 


Table 2a: Four-Year Trend for Scores in Online Organization & Design 


Category 

ROCS 

No. 

Indicator 

2007 

Fall 

2008 

Fall 

2010 

Fall 

2011 

Fall 


4 

Evidence of organization and clear navigation 

4.2 

3.9 

4.2 

3.9 


5 

Defined role of the online environment in the syllabus 

4.3 

4.6 

4.0 

3.8 

Online 

6 

Intentional page design and clarity of course information 

3.7 

3.1 

3.8 

3.4 


7 

Visual and functional consistency of Blackboard pages 

4.1 

3.4 

3.9 

3.1 


8 

Americans with Disabilities Act (ADA) accessibility 

3.1 

3.7 

3.6 

0.7 


Category Average 

3.9 

3.7 

3.9 

3.0 


Table 2b: Correlations between Scorers in Online Organization & Design in Fall2011 


Online Organization and Design (No Correlation) 

0.00 

4 

Evidence of organization and clear navigation 

-0.12 

5 

Defined role of the online environment in the syllabus 

0.32 

6 

Intentional page design and clarity of course information 

-0.21 

7 

Visual and functional consistency of Blackboard pages 

002 

8 

Americans with Disabilities Act (ADA) accessibility 

0 07 


Instructional Design & Delivery 

This category speaks directly to faculty innovation and inductive learning opportunities. Research 
indicates that a positive attitude is present when faculty is using technology in a meaningful and relevant way (Berge 
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et al., 2002). This finding is supported by prior research that indicates that technological competency is a positive 
motivator. 

Indicators in this category evaluate how innovative faculty members employ resources to create a quality 
online class. There was a strong correlation between team members’ scores. As Table 3a demonstrates, scores in 
this category fell considerably from the previous evaluation. The critical difference may again lie in the improved 
assessment methodology of evaluation for 2011. Results varied if evaluators used the syllabus as a reference for 
student outcomes versus Blackboard. As a solution, Hodges University created a syllabus database that maps 
program outcomes to the syllabus. This will allow for consistency in 2012 measures. 


Table 3a: Four-Year Trend for Scores in Instructional Design & Delivery 


Category 

ROCS 

No. 

Indicator 

2007 

Fall 

2008 

Fall 

2010 

Fall 

2011 

Fall 


9 

Interaction among student, instructor, and content 

3.4 

3.1 

4.2 

2.3 


10 

Alignment of course goals with learning outcomes 

4.0 

3.9 

4.6 

2.4 


11 

Integration of learning outcomes with instr. activities 

3.8 

4.0 

3.8 

2.1 

Instructional 
Design & 
Delivery 

12 

13 

14 

Multiple means of representation, action and expression, 
and engagement (i.e., Visual, textual, auditory, etc.) 

Critical thinking and problem-solving activities 

1-5 HU educ. outcomes are integrated and measurable 

2.9 

3.7 

2.9 

2.9 

3.6 

3.3 

4.0 

4.4 

4.4 

2.1 

2.6 

1.6 


15 

Academic rigor of course content and requirements 

4.1 

3.3 

4.2 

2.8 


16 

Course processes identified and communicated 

3.1 

3.4 

4.0 

2.1 


Category Average 

3.5 

3.4 

4.2 

2.2 


Table 3b: Correlations between Scorers in Instructional Design & Delivery in Fal l 2011 


Instructional Design and Delivery (Strong Correlation) 

0.72 

9 

Interaction among student, instructor, and content 

0.90 

10 

Alignment of course goals with learning outcomes 

0.81 

11 

Integration of learning outcomes with instructional activities 

0.81 

12 

Multiple means of representation, action and expression, and 

0.68 

engagement (i.e., Visual, textual, auditory, etc.) 


13 

Critical thinking and problem-solving activities 

0.49 

14 

HU's 5 educational outcomes are integrated and measurable 

0.41 

15 

Academic rigor of course content and requirements 

0.86 

16 

Course processes identified and communicated 

0.83 


Assessment & Evaluation 

Table 4b indicates a weak correlation and provides opportunities for team improvement. The lowest 
indicator, 21. Opportunities for self-assessment, reflection, and peer feedback is critical for the development of 
critical thinking and metacognition. This is an area that should show great improvement in 2012 due to university 
emphasis on the assessment process through both process improvement and professional development. 


Table 4a: Four-Year Trend for Scores in Assessment and Evaluation of SLO’s 


Category 

ROCS 

No. 

Indicator 

2007 

Fall 

2008 

Fall 

2010 

Fall 

2010 

Fall 


17 

Activities to assess student readiness for course 

content 

3.3 

4.0 

3.0 

3.3 


18 

Alignment of learning objectives/outcomes with 
instruction and assessment 

3.9 

3.4 

4.4 

4.1 

Assessment & 
Evaluation of 

19 

Multiple assessment strategies to measure 
knowledge, skills, attitudes 

3.9 

3.9 

4.2 

3.7 

SLOs 

20 

Timely feedback on student performance throughout 
the course 

3.3 

3.0 

4.1 

3.5 


21 

Opportunities for self-assessment, reflection, and 
peer feedback 

2.6 

3.6 

3.3 

2.8 


Category Average 

3.4 

3.6 

3.8 

3.5 
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Table 4b: Correlations between Scorers in Assessment & Evaluation in Fall 2011 


Assessment and Evaluation (Weak Correlation) 

0.16 

17 

Activities to assess student readiness for course content 

-0.26 

18 

Alignment of learning objectives/outcomes with instruction and 

0.07 

assessment 


19 

Multiple assessment strategies to measure knowledge, skills. 

0.36 

attitudes 


20 

Timely feedback on student performance throughout the course 

0.25 

21 

Opportunities for self-assessment, reflection, and peer feedback 

0.40 


Innovative Teaching with Technology 

As indicated in Table 5b, team scores for this category were moderately correlated. There is no doubt that 
faculty roles have been redefined by technology. Online courses can closely mirror traditional classroom delivery 
methods if the professor is innovative in his/her use of available technology (Schuster & Finkelstein, 2006). 
According to Tabata and Johnsrud (2008), quality instruction provided by innovative faculty is the only way any 
distance education initiative is successful. 

This review is not unlike reviews of previous years. The lowest two indicators were 22. Communication 
and learning through the use of Blackboard features and 23. Active learning and engagement facilitated by 
technology tools. Recent professional development has included Blackboard Collaborate, an upgrade of Wimba 
Classroom. The category should show improvement in 2012 measures. 


Table 5a: Four-Year Trend for Scores in Innovative Teaching with Technology 


Category 

ROCS 

No. 

Indicator 

2007 

Fall 

2008 

Fall 

2010 

Fall 

2011 

Fall 


22 

Communication and learning through use of 

Blackboard features 

2.8 

2.9 

3.3 

3.0 

Innovative 

23 

Active learning and engagement facilitated by 
technology tools 

2.4 

2.6 

3.3 

3.0 

Teaching with 
Technology 

24 

Relevance and variety of interactive activities (i.e., 

Web 2.0) 

2.1 

2.7 

3.1 

3.1 


25 

Use of Internet to engage students in the learning 
process 

2.9 

3.1 

3.5 

3.4 


Category Average 

2.6 

2.8 

3.3 

3.1 


Table 5b: Correlations between Scorers in Innovative Teaching with Technology in Fall 2011 


Innovative Teaching w/Technology (Moderate Correlation) 

0.56 

22 

Communication and learning through use of Blackboard features 

0.56 

23 

Active learning and engagement facilitated by technology tools 

0.20 

24 

Relevance and variety of interactive activities (i.e., Web 2.0) 

0.78 

25 

Use of Internet to engage students in the learning process 

0.71 


Faculty Use of Feedback 

Team scores in this category were weakly correlated but could be a function of the ability for evaluators to 
determine if feedback is being utilized. This is a training opportunity and could be as simple as improved 
documentation. The indicator that scored the lowest, 27. Opportunities for student feedback on Blackboard course 
delivery was improved with direction to create a discussion board entry for students to discuss issues with the 
professor. With greater emphasis upon the definition of each indicator, reliability should improve in 2012 measures. 
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Table 6a: Four-Year Trend for Scores in Faculty Use of Student Feedback 


Category 

No. 

Indicator 

2007 

Fall 

2008 

Fall 

2010 

Fall 

2011 

Fall 

Faculty 

Use of 
Student 

26 

Opportunities for student feedback on course content (e.g., 
learning outcomes, activities, and assessments). 

3.1 

3.3 

3.2 

3.6 

27 

Opportunities for student feedback on Blackboard course 
delivery 

2.4 

3.7 

2.9 

3.1 

Feedback 

28 

Use of formal and informal feedback to guide instruction/ 
Course Sequencing 

2.9 

3.4 

3.2 

3.2 



Category Average 

2.8 

3.5 

3.1 

3.3 


Table 6b: Correlations between Scorers in Faculty Use of Student Feedback in Fall 2011 


Faculty Use of Student Feedback (Weak Correlation) 

0.29 

26 

Opportunities for student feedback on course content 

0.40 

27 

Opportunities for student feedback on Blackboard course delivery 

0.15 

28 

Opportunities for student feedback on content sequencing 

0.32 


Improvements to evaluation process produced several notable results in 2011 and should improve 
significantly in 2012. Efforts by distance education staff produced some agreement on the application of quality 
indicators, resulting in consistent assessment by half of all evaluators. Improvements in team processes in the 2012 
will improve the reliability of the assessment process. An increased granularity was a benefit of the new process, in 
which 18 courses at the lower division, upper division, and graduate levels were evaluated. The new process 
employed by Hodges University solidified faculty as a solid learning community thus validating the conceptual 
concept of Orlikowski (2000) where structure is an attitude as opposed to physical. Additionally, the process closed 
the loop on reliability issues from previous evaluations and provided documentation for future training initiatives. 

CONCLUSION 

As noted earlier, this research is the baseline to evaluation of faculty innovation, technology acceptance, 
and use of inductive learning techniques to improve delivery of online course. The literature and analysis provided 
demonstrates the need for further research. Future initiatives include a more detailed review of inductive learning 
techniques supported by innovation and technology. 

According to research, faculty requires a strong support system to motivate them into actively participating 
in online learning. Universities that provide support through training, workshops, assessment, encouragement, and 
other forms produce significantly improved online courses (Chizman & Williams, 2001; Lee & Busch, 2005). This 
is further supported by research. A study by Tabata and Johnsrud (2008), revealed that there are eleven significant 
variables significant to faculty embracing online instruction. These include: 1) importance of using software, 2) 
importance of using e-resources, 3) skillful at using technology, 4) technology being important in professional work, 
5) faculty possessing technical skills, 6) quality of the online course being perceived as high as a traditional 
classroom course, 7) distance education being compatible to work style, 8) faculty self-image is enhanced by 
technological innovations, 9) distance education courses perceived as difficult, 10) the ability to see results of online 
delivery, and 11) the ability to try out online teaching before committing. 

It is noted throughout this research that institutional support is critical to the success of online instruction. 
In addition to the eleven variables listed above that indicate the probability of faculty members embracing online 
instruction, Tabata and Johnsrud (2008) found five variables that indicate when there is a decreased likelihood of 
faculty involvement in online instruction. These include the perception of faculty that: 1) resources are not available 
to support online teaching, 2) the institution does not value distance education, 3) participation in online instruction 
is not voluntary, 4) faculty cannot share online instruction results with other faculty, and 5) the advantages of 
teaching online do not outweigh the disadvantages. This research and results of the peer-to-peer evaluation 
underscore the importance of the support team at Hodges University. 

Hodges University is unique in its approach to distance education as faculty is afforded academic freedom 
similar to the traditional classroom. The administration is committed to supporting online instruction while 
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maintaining academic integrity through a detailed assessment program. Hodges University faculty are encouraged 
to professionally develop and supported through instructional design courses at the local and national levels. 
Technology tools such as Blackboard, Collaborate, WIMBA, Tegrity, Snagit, Jing, Voice podcasts and numerous 
others are readily available with training. With the exception of the basic university template for continuity, faculty 
members are free to be innovative and creative. The peer-to-peer review process assesses usage and provides both 
best practices and training opportunities. 

Future research will include a statistical analysis of technology acceptance, faculty innovation, and 
inductive learning techniques. This research will also be followed with a comparison to results from the 2012 study 
when it is available. Enhanced assessment policies and creation of a team approach provide an opportunity to 
compare 2011 and 2012 with the understanding that they were both evaluated with the same standards. This is an 
ongoing process that will serve to continuously improve online courses at Hodges University. 
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APPENDIX A 


Addendum'; 

Peer-to-Peer Review Trends 


Category 

No. 


2007 - Fall 

2008 - Fall 

2010 - Fall 

2011 - Fall 


Avg 

Max 

Min 

Avg 

Max 

Min 

Avg 

Max 

Min 

Avg 

Max 

Min 

Learner 

Support 

1 

Links to library and online learning 

resources 

3.7 

5 

2 

3.3 

4 

3 

3.7 

5 

0 

2.7 

5 

0 

2 

Course and program resources and contact 
information 

3.3 

4 

2 

3.4 

5 

2 

4.3 

5 

2 

2.9 

5 

0 

3 

Range of resources supporting course 

content 

3.1 

4 

2 

3.7 

5 

2 

3.9 

5 

0 

3.1 

5 

0 


Category Average 

Category Percentages 

3.4 

67% 

4.0 

80% 

2.7 

53% 

3.5 

70% 

4.7 

93% 

2.7 

53% 

3.9 

79% 

5 

100% 

0.7 

13% 

2.9 

58% 

5 

100% 

0 

0% 

Online 

Organization & 
Design 

4 

Evidence of organization and clear 
navigation 

4.2 

5 

3 

3.9 

5 

2 

4.2 

5 

2 

3.9 

5 

1 

5 

Defined role of the online environment in 
the syllabus 

4.3 

5 

3 

4.6 

5 

4 

4.0 

5 

2 

3.8 

5 

0 

6 

Intentional page design and clarity of 

course information 

3.7 

5 

2 

3.1 

5 

0 

3.8 

5 

0 

3.4 

5 

0 

7 

Visual and functional consistency of 
Blackboard pages 

4.1 

5 

3 

3.4 

5 

0 

3.9 

5 

0 

3.1 

5 

0 

8 

Americans with Disabilities Act (ADA) 
accessibility 

3.1 

4 

2 

3.7 

5 

3 

3.6 

5 

0 

0.7 

5 

0 


Category Average 
Category Percentages 

3.9 

78% 

4.6 

92% 

3.0 

60% 

3.7 

75% 

4.8 

96% 

2.0 

40% 

3.9 

78% 

5.0 

100% 

1.2 

24% 

3.0 

59% 

5.0 

100% 

0.0 

0% 

Instructional 
Design & 
Delivery 

9 

Interaction among student, instructor, and 
content 

3.4 

5 

1 

3.1 

5 

2 

4.2 

5 

2 

2.3 

4 

0 

10 

Alignment of course goals with learning 
outcomes 

4.0 

5 

3 

3.9 

5 

3 

4.6 

5 

4 

2.4 

5 

0 

11 

Integration of learning outcomes with 
instr. activities 

3.8 

5 

3 

4.0 

5 

3 

3.8 

5 

0 

2.1 

4 

0 

12 

Multiple means of representation, action 
and expression, and engagement (i.e.. 
Visual, textual, auditory, etc.) 

2.9 

4 

2 

2.9 

5 

1 

4.0 

5 

2 

2.1 

4 

0 

13 

Critical thinking and problem-solving 
activities 

3.7 

5 

1 

3.6 

5 

3 

4.4 

5 

2 

2.6 

4 

0 

14 

1-5 HU educ. outcomes are integrated and 

measurable 

2.9 

4 

2 

3.3 

4 

3 

4.4 

5 

3 

1.6 

4 

0 

15 

Academic rigor of course content and 
requirements 

4.1 

5 

2 

3.3 

5 

3 

4.2 

5 

0 

2.8 

5 

0 

16 

Course processes identified and 
communicated 

3.1 

4 

2 

3.4 

5 

3 

4.0 

5 

0 

2.1 

4 

0 


Category Average 
Category Percentages 

3.5 

70% 

4.1 

83% 

2.3 

45% 

3.4 

69% 

4.9 

98% 

3.0 

60% 

4.2 

84% 

5.0 

100% 

2.6 

53% 

2.2 

45% 

5.0 

100% 

0.0 

o% 

Assessment & 

17 

Activities to assess student readiness for 

course content 

3.3 

4 

2 

4.0 

5 

3 

3.0 

5 

0 

3.3 

5 

1 

18 

Alignment of learning objectives/outcomes 
with instruction and assessment 

3.9 

5 

2 

3.4 

5 

3 

4.4 

5 

3 

4.1 

5 

2 

19 

Multiple assessment strategies to measure 
knowledge, skills, attitudes 

3.9 

5 

1 

3.9 

5 

3 

4.2 

5 

3 

3.7 

5 

1 

SLOs 

20 

Timely feedback on student performance 
throughout the course 

3.3 

5 

1 

3.0 

5 

0 

4.1 

5 

2 

3.5 

5 

1 


21 

Opportunities for self-assessment, 
reflection, and peer feedback 

2.6 

4 

1 

3.6 

5 

3 

3.3 

5 

0 

2.8 

5 

1 

Category Average 
Category Percentages 

3.4 

68% 

4.6 

92% 

2.4 

48% 

3.6 

71% 

4.8 

96% 

2.8 

3.8 

76% 

5-0 

100% 

1,6 

32% 

3,5 

69% 

5.0 

100% 

1.0 

20% 


22 

Communication and learning through use 
of Blackboard features 

2.8 

4 

0 

2.9 

5 

1 

3.3 

5 

0 

3.0 

4 

1 

23 

Active learning and engagement facilitated 
by technology tools 

2.4 

4 

0 

2.6 

5 

0 

3.3 

5 

0 

3.0 

4 

2 

Teaching with 

24 

Relevance and variety of interactive 
activities (i.e., Web 2.0) 

2.1 

4 

0 

2.7 

5 

1 

3.1 

5 

0 

3.1 

5 

1 


25 

Use of Internet to engage students in the 
learning process 

2.9 

4 

0 

3.1 

5 

1 

3.5 

5 

0 

3.4 

5 

2 


Category Average 
Category Percentages 

2.6 

51% 

3.8 

75% 

0.0 

0% 

2.8 

56% 

5.0 

100% 

1.0 

20% 

3.3 

66% 

5.0 

100% 

0.5 3.1 

10% 63% 

5.0 

100% 

1.0 

20% 


Use of Student 

26 

Opportunities for student feedback on 
course content (e.g., learning outcomes, 
activities, and assessments). 

3.1 

5 

1 

3.3 

5 

1 

3.2 

5 

0 

3.6 

5 

1 

27 

Opportunities for student feedback on 

Blackboard course delivery 

2.4 

4 

0 

3.7 

5 

2 

2.9 

5 

0 

3.1 

5 

1 


28 

Use of formal and informal feedback to 
guide instruction/ Course Sequencing 

2.9 

4 

0 

3.4 

5 

? 

3.2 

5 

0 

3.2 

5 

1 


Category Average 

■Si 

4.0 

80% 

1-5 

30% 

3.5 

70% 

5.0 

100% 

1.7 

33% 

3.1 

62% 


0.0 

K 

3.3 

66%| 


1.0 

Grand Category Average 
Category Percentages 

3.2 

65% 

3.8 

76% 

2.2 

45% 

3.4 

68% 

4.9 

97% 

2.4 

47% 

3.7 

74% 

5.01 
100% 


2.9 

58% 

5.0 

100% 

0.0 

0% 


Source: Distance Education. (2011). Hodges University. Report on the peer-to-peer review. 
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